Neuro-response evaluated stimulus in virtual reality environments

ABSTRACT

A system presents stimulus materials such as products, product packages, displays, services, offerings, etc., in virtual reality environments such as market aisles, store shelves, showroom floors, etc. Sensory experiences output to the user via the virtual reality environment elicit user interactivity. User activity and responses are used to modify marketing materials and/or virtual reality environments. Neuro-response data including electroencephalography (EEG) data is collected from users in order to evaluate the effectiveness of marketing materials in virtual reality environments. In particular examples, neuro-response data is used to modify marketing materials and virtual reality environments presented to the user.

RELATED APPLICATIONS

This patent is related to U.S. patent application Ser. No. 12/056,190;U.S. patent application Ser. No. 12/056,211; U.S. patent applicationSer. No. 12/056,221; U.S. patent application Ser. No. 12/056,225; U.S.patent application Ser. No. 12/113,863; U.S. patent application Ser. No.12/113,870; U.S. patent application Ser. No. 12/122,240; U.S. patentapplication Ser. No. 12/122,253; U.S. patent application Ser. No.12/122,262; U.S. patent application Ser. No. 12/135,066; U.S. patentapplication Ser. No. 12/135,074; U.S. patent application Ser. No.12/182,851; U.S. patent application Ser. No. 12/182,874; U.S. patentapplication Ser. No. 12/199,557; U.S. patent application Ser. No.12/199,583; U.S. patent application Ser. No. 12/199,596; U.S. patentapplication Ser. No. 12/200,813; U.S. patent application Ser. No.12/234,372; U.S. patent application Ser. No. 12/135,069; U.S. patentapplication Ser. No. 12/234,388; U.S. patent application Ser. No.12/544,921; U.S. patent application Ser. No. 12/544,934; U.S. patentapplication Ser. No. 12/546,586; U.S. patent application Ser. No.12/544,958; U.S. patent application Ser. No. 12/846,242; U.S. patentapplication Ser. No. 12/410,380; U.S. patent application Ser. No.12/410,372; U.S. patent application Ser. No. 12/413,297; U.S. patentapplication Ser. No. 12/545,455; U.S. patent application Ser. No.12/608,660; U.S. patent application Ser. No. 12/608,685; U.S. patentapplication Ser. No. 13/444,149; U.S. patent application Ser. No.12/608,696; U.S. patent application Ser. No. 12/731,868; U.S. patentapplication Ser. No. 13/045,457; U.S. patent application Ser. No.12/778,810; U.S. patent application Ser. No. 12/778,828; U.S. patentapplication Ser. No. 13/104,821; U.S. patent application Ser. No.13/104,840; U.S. patent application Ser. No. 12/884,034; U.S. patentapplication Ser. No. 12/868,531; U.S. patent application Ser. No.12/913,102; U.S. patent application Ser. No. 12/853,213; and U.S. patentapplication Ser. No. 13/105,774.

TECHNICAL FIELD

The present disclosure relates to using neuro-response data to evaluatemarketing and entertainment in virtual reality environments.

DESCRIPTION OF RELATED ART

Conventional systems for evaluating marketing materials typicallyinvolve monitoring and surveying individuals exposed to materials suchas products, packages, advertisements, and services. Attempts have beenmade to present marketing materials in their natural environments suchas showrooms, store shelves, displays, etc. However, mechanisms forpresenting marketing materials in natural environments are limited. Insome examples, individuals are asked to respond to surveys quickly afterexposure to marketing materials in actual environments, but informationcollected is typically limited. Furthermore, conventional systems aresubject to brain pattern, semantic, syntactic, metaphorical, cultural,and interpretive errors that prevent accurate and repeatable analyses.

Consequently, it is desirable to provide improved methods and apparatusfor evaluating marketing materials in natural environments that useneuro-response data such as central nervous system, autonomic nervoussystem, and effector system measurements along with survey based data.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings, whichillustrate particular example embodiments.

FIGS. 1A-1B illustrate a particular example of a system for evaluatingstimulus material in a virtual reality environment.

FIGS. 2A-2E illustrate a particular example of a neuro-response datacollection mechanism.

FIG. 3 illustrates examples of data models that can be used with astimulus and response repository.

FIG. 4 illustrates one example of a query that can be used with theneuro-response collection system.

FIG. 5 illustrates one example of a report generated using theneuro-response collection system.

FIG. 6 illustrates one example of a technique for evaluating stimulusmaterial in a virtual reality environment.

FIG. 7 provides one example of a system that can be used to implementone or more mechanisms.

DESCRIPTION OF PARTICULAR EMBODIMENTS

Reference will now be made in detail to some specific examples of theinvention including the best modes contemplated by the inventors forcarrying out the invention. Examples of these specific embodiments areillustrated in the accompanying drawings. While the invention isdescribed in conjunction with these specific embodiments, it will beunderstood that it is not intended to limit the invention to thedescribed embodiments. On the contrary, it is intended to coveralternatives, modifications, and equivalents as may be included withinthe spirit and scope of the invention as defined by the appended claims.

For example, the techniques and mechanisms of the present invention willbe described in the context of particular types of stimulus materials.However, it should be noted that the techniques and mechanisms of thepresent invention apply to a variety of different types of stimulusmaterials including marketing and entertainment materials. It should benoted that various mechanisms and techniques can be applied to any typeof stimuli. In the following description, numerous specific details areset forth in order to provide a thorough understanding of the presentinvention. Particular example embodiments of the present invention maybe implemented without some or all of these specific details. In otherinstances, well known process operations have not been described indetail in order not to unnecessarily obscure the present invention.

Various techniques and mechanisms of the present invention willsometimes be described in singular form for clarity. However, it shouldbe noted that some embodiments include multiple iterations of atechnique or multiple instantiations of a mechanism unless notedotherwise. For example, a system uses a processor in a variety ofcontexts. However, it will be appreciated that a system can use multipleprocessors while remaining within the scope of the present inventionunless otherwise noted. Furthermore, the techniques and mechanisms ofthe present invention will sometimes describe a connection between twoentities. It should be noted that a connection between two entities doesnot necessarily mean a direct, unimpeded connection, as a variety ofother entities may reside between the two entities. For example, aprocessor may be connected to memory, but it will be appreciated that avariety of bridges and controllers may reside between the processor andmemory. Consequently, a connection does not necessarily mean a direct,unimpeded connection unless otherwise noted.

Overview

A system presents stimulus materials such as products, product packages,displays, services, offerings, etc., in virtual reality environmentssuch as market aisles, store shelves, showroom floors, etc. Sensoryexperiences output to the user via the virtual reality environmentelicit user interactivity. User activity and responses are used tomodify marketing materials and/or virtual reality environments.Neuro-response data including electroencephalography (EEG) data iscollected from users in order to evaluate the effectiveness of marketingmaterials in virtual reality environments. In particular examples,neuro-response data is used to modify marketing materials and virtualreality environments presented to the user.

Example Embodiments

Marketing materials such as products, product packages, brochures,displays, signs, offerings, and arrangements are typically evaluated bysurveying individuals exposed to the marketing materials. Surveyresponses and focus groups elicit user opinions about the marketingmaterials. The survey responses and focus groups provide some limitedinformation about the effectiveness of the marketing materials. It isrecognized that user responses to marketing materials in a laboratory orevaluation setting can sometimes be different than user responses to themarketing materials in a natural environment, such as a store shelf, asupermarket aisle, a tradeshow floor, building, or a showroom. However,opportunities to evaluate the effectiveness of marketing materials innatural environments are limited.

In some instances, efforts are made to elicit user responses tomarketing materials after users visit actual showrooms, stores, ortradeshows. In other instances, model stores, displays, and mockpresentations are set up to test the effectiveness of particularpackages, displays, presentations, offerings, etc. However, using actualdisplays or establishing mock presentations is cumbersome andinflexible. It is highly inefficient to test a variety of presentationsor make changes to presentations based on user feedback. Furthermore,even when ample user feedback is obtained, user feedback is subject tobrain pattern, semantic, syntactic, metaphorical, cultural, andinterpretive errors that prevent accurate and repeatable analyses.

Consequently, the techniques of the present invention provide mechanismsfor evaluating marketing materials presented in virtual realityenvironments by using neuro-response data. In some examples,neuro-response data along with other survey and focus group data is usedto test marketing presentations and displays in virtual realityenvironments. Virtual reality environments and virtual realityenvironment templates can be generated to allow reuse, customization,and integration with generated marketing materials.

Neuro-response data is analyzed to determine the effectiveness ofmarketing materials presented in various virtual reality environments toparticular individuals. Individuals are provided with mechanisms tointeract with the virtual reality environment and marketing materialsare manipulated in the framework of the virtual reality environment.Sensors, cameras, microphones, motion detectors, gyroscopes, temperaturesensors, etc., can all be used to monitor user responses to allow notonly manipulation of the virtual reality environment but modification ofthe marketing materials presented. In particular embodiments,neuro-response data is used to evaluate the effectiveness of marketingmaterials and make real-time adjustments and modifications to marketingmaterials or the virtual reality environment presented.

Neuro-response measurements such as central nervous system, autonomicnervous system, and effector measurements can be used to evaluatesubjects during stimulus presentation. Some examples of central nervoussystem measurement mechanisms include Functional Magnetic ResonanceImaging (fMRI), Electroencephalography (EEG), Magnetoencephlography(MEG), and Optical Imaging. Optical imaging can be used to measure theabsorption or scattering of light related to concentration of chemicalsin the brain or neurons associated with neuronal firing. MEG measuresmagnetic fields produced by electrical activity in the brain. fMRImeasures blood oxygenation in the brain that correlates with increasedneural activity. However, current implementations of fMRI have poortemporal resolution of few seconds. EEG measures electrical activityassociated with post synaptic currents occurring in the millisecondsrange. Subcranial EEG can measure electrical activity with the mostaccuracy, as the bone and dermal layers weaken transmission of a widerange of frequencies. Nonetheless, surface EEG provides a wealth ofelectrophysiological information if analyzed properly. Even portable EEGwith dry electrodes provides a large amount of neuro-responseinformation.

Autonomic nervous system measurement mechanisms includeElectrocardiograms (EKG) and pupillary dilation, etc. Effectormeasurement mechanisms include Electrooculography (EOG), eye tracking,facial emotion encoding, reaction time etc.

Multiple modes and manifestations of precognitive neural signatures areblended with cognitive neural signatures and post cognitiveneurophysiological manifestations to more accurately performneuro-response analysis. In some examples, autonomic nervous systemmeasures are themselves used to validate central nervous systemmeasures. Effector and behavior responses are blended and combined withother measures. According to various embodiments, central nervoussystem, autonomic nervous system, and effector system measurements areaggregated into a measurement that allows evaluation of stimulusmaterial effectiveness in particular environments.

In particular embodiments, subjects are exposed to stimulus material anddata such as central nervous system, autonomic nervous system, andeffector data is collected during exposure. According to variousembodiments, data is collected in order to determine a resonance measurethat aggregates multiple component measures that assess resonance data.In particular embodiments, specific event related potential (ERP)analyses and/or event related power spectral perturbations (ERPSPs) areevaluated for different regions of the brain both before a subject isexposed to stimulus and each time after the subject is exposed tostimulus.

According to various embodiments, pre-stimulus and post-stimulusdifferential as well as target and distracter differential measurementsof ERP time domain components at multiple regions of the brain aredetermined (DERP). Event related time-frequency analysis of thedifferential response to assess the attention, emotion and memoryretention (DERPSPs) across multiple frequency bands including but notlimited to theta, alpha, beta, gamma and high gamma is performed. Inparticular embodiments, single trial and/or averaged DERP and/or DERPSPscan be used to enhance the resonance measure and determine priminglevels for various products and services.

According to various embodiments, enhanced neuro-response data isgenerated using a data analyzer that performs both intra-modalitymeasurement enhancements and cross-modality measurement enhancements.According to various embodiments, brain activity is measured not just todetermine the regions of activity, but to determine interactions andtypes of interactions between various regions. The techniques andmechanisms of the present invention recognize that interactions betweenneural regions support orchestrated and organized behavior. Attention,emotion, memory, and other abilities are not merely based on one part ofthe brain but instead rely on network interactions between brainregions.

The techniques and mechanisms of the present invention further recognizethat different frequency bands used for multi-regional communication canbe indicative of the effectiveness of stimuli. In particularembodiments, evaluations are calibrated to each subject and synchronizedacross subjects. In particular embodiments, templates are created forsubjects to create a baseline for measuring pre and post stimulusdifferentials. According to various embodiments, stimulus generators areintelligent and adaptively modify specific parameters such as exposurelength and duration for each subject being analyzed.

A variety of modalities can be used including EEG, GSR, EKG, pupillarydilation, EOG, eye tracking, facial emotion encoding, reaction time,etc. Individual modalities such as EEG are enhanced by intelligentlyrecognizing neural region communication pathways. Cross modalityanalysis is enhanced using a synthesis and analytical blending ofcentral nervous system, autonomic nervous system, and effectorsignatures. Synthesis and analysis by mechanisms such as time and phaseshifting, correlating, and validating intra-modal determinations allowgeneration of a composite output characterizing the significance ofvarious data responses.

According to various embodiments, survey based and actual expressedresponses and actions for particular groups of users are integrated withneuro-response data and stored in a stimulus material and virtualreality environment. According to particular embodiments,pre-articulation predictions of expressive response for various stimulusmaterial can be made by analyzing neuro-response data.

FIG. 1A illustrates one example of a system for present marketingmaterials in virtual reality environments. According to variousembodiments, a stimulus material generator 101 constructs products,product packages, displays, labels, boxes, signs, offerings, andadvertising using custom or template designs. In particular examples,companies, firms, and individuals wanting to test marketing materialsprovide the designs that can be incorporated into designs or wireframesfor use in virtual reality environments. A virtual reality environmentgenerator 103 can similarly construct custom or template based designsfor store aisles, shopping malls, showrooms, tradeshow floors, etc.,that can be integrated with marketing materials 101 using a stimulusmaterial and virtual reality integration mechanism 111.

The integrated marketing materials and virtual reality environment areprovided to a presentation device 121. The presentation device 121 mayinclude screens, headsets, domes, multidimensional displays, speakers,motion simulation devices, movable platforms, smell generators, etc., toprovide the subject 123 with a simulated environment. Subject responsecollection mechanism 131 may include cameras recorders, motiondetectors, etc., that capture subject activity and responses. Accordingto various embodiments, neuro-response data collection mechanisms arealso used to capture neuro-response data such as electroencephalography(EEG) data for the subject presented with stimulus materials. Inparticular embodiments, feedback and modification mechanism 141 usessubject responses to modify marketing materials and/or the virtualreality environment based on subject actions. According to variousembodiments, product packages may be manipulated by a subject in thevirtual reality environment. In particular embodiments, store displaysmay be viewed from different angles, products may be opened, etc.

According to various embodiments, neuro-response data including EEG datais used to make real-time modifications to marketing materials andvirtual reality environments. In particular embodiments, lack ofinterest is detected using neuro-response data and different marketingmaterials are dynamically presented to the user as the user moves alongin a grocery aisle.

FIG. 1B illustrates one example of a neuro-response data collectionmechanism that can be used with users exposed to stimulus material invirtual reality environments. According to various embodiments, thevirtual reality stimulus presentation includes a stimulus presentationin virtual reality environments device 151. In particular embodiments,the virtual reality environment presentation device 151 is merely adisplay, monitor, screen, etc., that displays stimulus material in thecontext of a virtual reality environment to a user. The stimulusmaterial may be a product, product package, service, offering,advertisement, placard, brochure, etc., placed in the context of asupermarket aisle, convenience store, room, etc.

The stimuli can involve a variety of senses and occur with or withouthuman supervision. Continuous and discrete modes are supported.According to various embodiments, the virtual reality environmentpresentation device 151 also has protocol generation capability to allowintelligent customization of stimulus and environments provided tomultiple subjects in different settings such as laboratory, corporate,and home settings.

According to various embodiments, virtual reality environmentpresentation device 151 could include devices such as headsets, goggles,projection systems, display devices, speakers, tactile surfaces, etc.,for presenting the stimulus in virtual reality environments.

According to various embodiments, the subjects 153 are connected to datacollection devices 155. The data collection devices 155 may include avariety of neuro-response measurement mechanisms including neurologicaland neurophysiological measurements systems such as EEG, EOG, MEG,pupillary dilation, eye tracking, facial emotion encoding, and reactiontime devices, etc. According to various embodiments, neuro-response dataincludes central nervous system, autonomic nervous system, and effectordata. In particular embodiments, the data collection devices 155 includeEEG 161, EOG 163, and fMRI 165. In some instances, only a single datacollection device is used. Data collection may proceed with or withouthuman supervision.

The data collection device 155 collects neuro-response data frommultiple sources. This includes a combination of devices such as centralnervous system sources (EEG), autonomic nervous system sources (EKG,pupillary dilation), and effector sources (EOG, eye tracking, facialemotion encoding, reaction time). In particular embodiments, datacollected is digitally sampled and stored for later analysis. Inparticular embodiments, the data collected could be analyzed inreal-time. According to particular embodiments, the digital samplingrates are adaptively chosen based on the neurophysiological andneurological data being measured.

In one particular embodiment, the system includes EEG 161 measurementsmade using scalp level electrodes, EOG 163 measurements made usingshielded electrodes to track eye data, fMRI 165 measurements performedusing a differential measurement system, a facial muscular measurementthrough shielded electrodes placed at specific locations on the face,and a facial affect graphic and video analyzer adaptively derived foreach individual.

In particular embodiments, the data collection devices are clocksynchronized with a virtual reality environment presentation device 151.In particular embodiments, the data collection devices 155 also includea condition evaluation subsystem that provides auto triggers, alerts andstatus monitoring and visualization components that continuously monitorthe status of the subject, data being collected, and the data collectioninstruments. The condition evaluation subsystem may also present visualalerts and automatically trigger remedial actions. According to variousembodiments, the data collection devices include mechanisms for not onlymonitoring subject neuro-response to stimulus materials, but alsoinclude mechanisms for identifying and monitoring the stimulusmaterials. For example, data collection devices 155 may be synchronizedwith a set-top box to monitor channel changes. In other examples, datacollection devices 155 may be directionally synchronized to monitor whena subject is no longer paying attention to stimulus material. In stillother examples, the data collection devices 155 may receive and storestimulus material generally being viewed by the subject, whether thestimulus is a program, a commercial, printed material, or a sceneoutside a window. The data collected allows analysis of neuro-responseinformation and correlation of the information to actual stimulusmaterial and not mere subject distractions.

According to various embodiments, the virtual reality stimuluspresentation system also includes a data cleanser device 171. Inparticular embodiments, the data cleanser device 171 filters thecollected data to remove noise, artifacts, and other irrelevant datausing fixed and adaptive filtering, weighted averaging, advancedcomponent extraction (like PCA, ICA), vector and component separationmethods, etc. This device cleanses the data by removing both exogenousnoise (where the source is outside the physiology of the subject, e.g. aphone ringing while a subject is viewing a video) and endogenousartifacts (where the source could be neurophysiological, e.g. musclemovements, eye blinks, etc.).

The artifact removal subsystem includes mechanisms to selectivelyisolate and review the response data and identify epochs with timedomain and/or frequency domain attributes that correspond to artifactssuch as line frequency, eye blinks, and muscle movements. The artifactremoval subsystem then cleanses the artifacts by either omitting theseepochs, or by replacing these epoch data with an estimate based on theother clean data (for example, an EEG nearest neighbor weightedaveraging approach).

According to various embodiments, the data cleanser device 171 isimplemented using hardware, firmware, and/or software. It should benoted that although a data cleanser device 171 is shown located after adata collection device 155, the data cleanser device 171 like othercomponents may have a location and functionality that varies based onsystem implementation. For example, some systems may not use anyautomated data cleanser device whatsoever while in other systems, datacleanser devices may be integrated into individual data collectiondevices.

In particular embodiments, a survey and interview system collects andintegrates user survey and interview responses to combine withneuro-response data to more effectively perform virtual reality stimuluspresentation. According to various embodiments, the survey and interviewsystem obtains information about user characteristics such as age,gender, income level, location, interests, buying preferences, hobbies,etc.

According to various embodiments, the virtual reality stimuluspresentation system includes a data analyzer 173 associated with thedata cleanser 171. The data analyzer 173 uses a variety of mechanisms toanalyze underlying data in the system to determine resonance. Accordingto various embodiments, the data analyzer 173 customizes and extractsthe independent neurological and neuro-physiological parameters for eachindividual in each modality, and blends the estimates within a modalityas well as across modalities to elicit an enhanced response to thepresented stimulus material. In particular embodiments, the dataanalyzer 173 aggregates the response measures across subjects in adataset.

According to various embodiments, neurological and neuro-physiologicalsignatures are measured using time domain analyses and frequency domainanalyses. Such analyses use parameters that are common acrossindividuals as well as parameters that are unique to each individual.The analyses could also include statistical parameter extraction andfuzzy logic based attribute estimation from both the time and frequencycomponents of the synthesized response.

In some examples, statistical parameters used in a blended effectivenessestimate include evaluations of skew, peaks, first and second moments,distribution, as well as fuzzy estimates of attention, emotionalengagement and memory retention responses.

According to various embodiments, the data analyzer 173 may include anintra-modality response synthesizer and a cross-modality responsesynthesizer. In particular embodiments, the intra-modality responsesynthesizer is configured to customize and extract the independentneurological and neurophysiological parameters for each individual ineach modality and blend the estimates within a modality analytically toelicit an enhanced response to the presented stimuli. In particularembodiments, the intra-modality response synthesizer also aggregatesdata from different subjects in a dataset.

According to various embodiments, the cross-modality responsesynthesizer or fusion device blends different intra-modality responses,including raw signals and signals output. The combination of signalsenhances the measures of effectiveness within a modality. Thecross-modality response fusion device can also aggregate data fromdifferent subjects in a dataset.

According to various embodiments, the data analyzer 173 also includes acomposite enhanced effectiveness estimator (CEEE) that combines theenhanced responses and estimates from each modality to provide a blendedestimate of the effectiveness. In particular embodiments, blendedestimates are provided for each exposure of a subject to stimulusmaterials. The blended estimates are evaluated over time to assessresonance characteristics. According to various embodiments, numericalvalues are assigned to each blended estimate. The numerical values maycorrespond to the intensity of neuro-response measurements, thesignificance of peaks, the change between peaks, etc. Higher numericalvalues may correspond to higher significance in neuro-responseintensity. Lower numerical values may correspond to lower significanceor even insignificant neuro-response activity. In other examples,multiple values are assigned to each blended estimate. In still otherexamples, blended estimates of neuro-response significance aregraphically represented to show changes after repeated exposure.

According to various embodiments, a data analyzer 173 passes data to aresonance estimator that assesses and extracts resonance patterns. Inparticular embodiments, the resonance estimator determines entitypositions in various stimulus segments and matches position informationwith eye tracking paths while correlating saccades with neuralassessments of attention, memory retention, and emotional engagement. Inparticular embodiments, the resonance estimator stores data in thepriming repository system. As with a variety of the components in thesystem, various repositories can be co-located with the rest of thesystem and the user, or could be implemented in remote locations.

Data from various repositories is blended and passed to a virtualreality stimulus presentation engine to generate patterns, responses,and predictions 175. In some embodiments, the virtual reality stimuluspresentation engine compares patterns and expressions associated withprior users to predict expressions of current users. According tovarious embodiments, patterns and expressions are combined withorthogonal survey, demographic, and preference data. In particularembodiments linguistic, perceptual, and/or motor responses are elicitedand predicted. Response expression selection and pre-articulationprediction of expressive responses are also evaluated.

FIGS. 2A-2E illustrate a particular example of a neuro-response datacollection mechanism. FIG. 2A shows a perspective view of aneuro-response data collection mechanism including multiple dryelectrodes. According to various embodiments, the neuro-response datacollection mechanism is a headset having point or teeth electrodesconfigured to contact the scalp through hair without the use ofelectro-conductive gels. In particular embodiments, each electrode isindividually amplified and isolated to enhance shielding androutability. In some examples, each electrode has an associatedamplifier implemented using a flexible printed circuit. Signals may berouted to a controller/processor for immediate transmission to a dataanalyzer or stored for later analysis. A controller/processor may beused to synchronize neuro-response data with stimulus materials. Theneuro-response data collection mechanism may also have receivers forreceiving clock signals and processing neuro-response signals. Theneuro-response data collection mechanisms may also have transmitters fortransmitting clock signals and sending data to a remote entity such as adata analyzer.

FIGS. 2B-2E illustrate top, side, rear, and perspective views of theneuro-response data collection mechanism. The neuro-response datacollection mechanism includes multiple electrodes including right sideelectrodes 261 and 263, left side electrodes 221 and 223, frontelectrodes 231 and 233, and rear electrode 251. It should be noted thatspecific electrode arrangement may vary from implementation toimplementation. However, the techniques and mechanisms of the presentinvention avoid placing electrodes on the temporal region to preventcollection of signals generated based on muscle contractions. Avoidingcontact with the temporal region also enhances comfort during sustainedwear.

According to various embodiments, forces applied by electrodes 221 and223 counterbalance forces applied by electrodes 261 and 263. Inparticular embodiments, forces applied by electrodes 231 and 233counterbalance forces applied by electrode 251. In particularembodiments, the EEG dry electrodes operate to detect neurologicalactivity with minimal interference from hair and without use of anyelectrically conductive gels. According to various embodiments,neuro-response data collection mechanism also includes EOG sensors suchas sensors used to detect eye movements.

According to various embodiments, data acquisition using electrodes 221,223, 231, 233, 251, 261, and 263 is synchronized with stimulus materialpresented to a user. Data acquisition can be synchronized with stimulusmaterial presented by using a shared clock signal. The shared clocksignal may originate from the stimulus material presentation mechanism,a headset, a cell tower, a satellite, etc. The data collection mechanism201 also includes a transmitter and/or receiver to send collectedneuro-response data to a data analysis system and to receive clocksignals as needed. In some examples, a transceiver transmits allcollected media such as video and/or audio, neuro-response, and sensordata to a data analyzer. In other examples, a transceiver transmits onlyinteresting data provided by a filter. According to various embodiments,neuro-response data is correlated with timing information for stimulusmaterial presented to a user.

In some examples, the transceiver can be connected to a computer systemthat then transmits data over a wide area network to a data analyzer. Inother examples, the transceiver sends data over a wide area network to adata analyzer. Other components such as fMRI and MEG that are not yetportable but may become portable at some point may also be integratedinto a headset.

It should be noted that some components of a neuro-response datacollection mechanism have not been shown for clarity. For example, abattery may be required to power components such as amplifiers andtransceivers. Similarly, a transceiver may include an antenna that issimilarly not shown for clarity purposes. It should also be noted thatsome components are also optional. For example, filters or storage maynot be required.

FIG. 3 illustrates examples of data models that can be used for storageof information associated with collection of neuro-response data.According to various embodiments, a dataset data model 301 includes aname 303 and/or identifier, client attributes 305, a subject pool 307,logistics information 309 such as the location, date, and stimulusmaterial 311 identified using user entered information or video andaudio detection.

In particular embodiments, a subject attribute data model 315 includes asubject name 317 and/or identifier, contact information 321, anddemographic attributes 319 that may be useful for review of neurologicaland neuro-physiological data. Some examples of pertinent demographicattributes include marriage status, employment status, occupation,household income, household size and composition, ethnicity, geographiclocation, sex, race. Other fields that may be included in data model 315include shopping preferences, entertainment preferences, and financialpreferences. Shopping preferences include favorite stores, shoppingfrequency, categories shopped, favorite brands. Entertainmentpreferences include network/cable/satellite access capabilities,favorite shows, favorite genres, and favorite actors. Financialpreferences include favorite insurance companies, preferred investmentpractices, banking preferences, and favorite online financialinstruments. A variety of subject attributes may be included in asubject attributes data model 315 and data models may be preset orcustom generated to suit particular purposes.

Other data models may include a data collection data model 337.According to various embodiments, the data collection data model 337includes recording attributes 339, equipment identifiers 341, modalitiesrecorded 343, and data storage attributes 345. In particularembodiments, equipment attributes 341 include an amplifier identifierand a sensor identifier.

Modalities recorded 343 may include modality specific attributes likeEEG cap layout, active channels, sampling frequency, and filters used.EOG specific attributes include the number and type of sensors used,location of sensors applied, etc. Eye tracking specific attributesinclude the type of tracker used, data recording frequency, data beingrecorded, recording format, etc. According to various embodiments, datastorage attributes 345 include file storage conventions (format, namingconvention, dating convention), storage location, archival attributes,expiry attributes, etc.

A preset query data model 349 includes a query name 351 and/oridentifier, an accessed data collection 353 such as data segmentsinvolved (models, databases/cubes, tables, etc.), access securityattributes 355 included who has what type of access, and refreshattributes 357 such as the expiry of the query, refresh frequency, etc.Other fields such as push-pull preferences can also be included toidentify an auto push reporting driver or a user driven report retrievalsystem.

FIG. 4 illustrates examples of queries that can be performed to obtaindata associated with neuro-response data collection. According tovarious embodiments, queries are defined from general or customizedscripting languages and constructs, visual mechanisms, a library ofpreset queries, diagnostic querying including drill-down diagnostics,and eliciting what if scenarios. According to various embodiments,subject attributes queries 415 may be configured to obtain data from aneuro-informatics repository using a location 417 or geographicinformation, session information 421 such as timing information for thedata collected. Location information 423 may also be collected. In someexamples, a neuro-response data collection mechanism includes GPS orother location detection mechanisms. Demographics attributes 419 includehousehold income, household size and status, education level, age ofkids, etc.

Other queries may retrieve stimulus material recorded based on shoppingpreferences of subject participants, countenance, physiologicalassessment, completion status. For example, a user may query for dataassociated with product categories, products shopped, shops frequented,subject eye correction status, color blindness, subject state, signalstrength of measured responses, alpha frequency band ringers, musclemovement assessments, segments completed, etc.

Response assessment based queries 437 may include attention scores 439,emotion scores, 441, retention scores 443, and effectiveness scores 445.Such queries may obtain materials that elicited particular scores.Response measure profile based queries may use mean measure thresholds,variance measures, number of peaks detected, etc. Group response queriesmay include group statistics like mean, variance, kurtosis, p-value,etc., group size, and outlier assessment measures. Still other queriesmay involve testing attributes like test location, time period, testrepetition count, test station, and test operator fields. A variety oftypes and combinations of types of queries can be used to efficientlyextract data.

FIG. 5 illustrates examples of reports that can be generated. Accordingto various embodiments, client assessment summary reports 501 includeeffectiveness measures 503, component assessment measures 505, andneuro-response data collection measures 507. Effectiveness assessmentmeasures include composite assessment measure(s),industry/category/client specific placement (percentile, ranking, etc.),actionable grouping assessment such as removing material, modifyingsegments, or fine tuning specific elements, etc, and the evolution ofthe effectiveness profile over time. In particular embodiments,component assessment reports include component assessment measures likeattention, emotional engagement scores, percentile placement, ranking,etc. Component profile measures include time based evolution of thecomponent measures and profile statistical assessments. According tovarious embodiments, reports include the number of times material isassessed, attributes of the multiple presentations used, evolution ofthe response assessment measures over the multiple presentations, andusage recommendations.

According to various embodiments, client cumulative reports 511 includemedia grouped reporting 513 of all stimulus assessed, campaign groupedreporting 515 of stimulus assessed, and time/location grouped reporting517 of stimulus assessed. According to various embodiments, industrycumulative and syndicated reports 521 include aggregate assessmentresponses measures 523, top performer lists 525, bottom performer lists527, outliers 529, and trend reporting 531. In particular embodiments,tracking and reporting includes specific products, categories,companies, brands.

FIG. 6 illustrates one example of evaluation of stimulus presentation ina virtual reality environment. At 601, user information is received froma subject provided with a neuro-response data collection mechanism.According to various embodiments, the subject provides data includingage, gender, income, location, interest, ethnicity, etc. At 603,stimulus material is received. In particular embodiments, stimulusmaterial is received from companies, firms, individuals, etc., seekingto evaluate their products, product labels, displays, brochures,services, offerings, etc., in a virtual reality environment. Inparticular examples, stimulus material is dynamically generated usinginformation provided by advertisers. According to various embodiments, avirtual reality environment appropriate for the stimulus material isselected and customized at 605. Virtual reality environments may includesuper market aisles, shopping centers, store shelves, showrooms, tradeshow floors, offices, etc.

At 607, stimulus material is integrated into a virtual realityenvironment and presented to a user. At 609, interaction data isreceived from users exposed to stimulus material. Interaction data maybe received from haptic gloves, platforms, sensors, cameras,microphones, platforms, magnetic fields, controllers, etc.

At 611, neuro-response data is received from the subject neuro-responsedata collection mechanism. In some particular embodiments, EEG, EOG,pupillary dilation, facial emotion encoding data, video, images, audio,GPS data, etc., can all be transmitted from the subject to aneuro-response data analyzer. In particular embodiments, only EEG datais transmitted. At 613, stimulus material and the virtual realityenvironment is modified based on user interaction. In particularembodiments, products may be manipulated by the user in the virtualreality environment. According to various embodiments, stimulus materialand/or the virtual reality environment can also be modified based onneuro-response data at 615. In particular embodiments, if a user isdetermined to be losing interest in a product, a different product maybe presented. Alternatively, a different environment displaying theproduct may be presented after a transition from one store to another.According to various embodiments, neuro-response and associated data istransmitted directly from an EEG cap wide area network interface to adata analyzer. In particular embodiments, neuro-response and associateddata is transmitted to a computer system that then performs compressionand filtering of the data before transmitting the data to a dataanalyzer over a network.

According to various embodiments, data is also passed through a datacleanser to remove noise and artifacts that may make data more difficultto interpret. According to various embodiments, the data cleanserremoves EEG electrical activity associated with blinking and otherendogenous/exogenous artifacts. Data cleansing may be performed beforeor after data transmission to a data analyzer.

At 617, neuro-response data is synchronized with timing, environment,and other stimulus material data. In particular embodiments,neuro-response data is synchronized with a shared clock source.According to various embodiments, neuro-response data such as EEG andEOG data is tagged to indicate what the subject is viewing or listeningto at a particular time.

At 619, data analysis is performed. Data analysis may includeintra-modality response synthesis and cross-modality response synthesisto enhance effectiveness measures. It should be noted that in someparticular instances, one type of synthesis may be performed withoutperforming other types of synthesis. For example, cross-modalityresponse synthesis may be performed with or without intra-modalitysynthesis.

A variety of mechanisms can be used to perform data analysis 609. Inparticular embodiments, a stimulus attributes repository is accessed toobtain attributes and characteristics of the stimulus materials, alongwith purposes, intents, objectives, etc. In particular embodiments, EEGresponse data is synthesized to provide an enhanced assessment ofeffectiveness. According to various embodiments, EEG measures electricalactivity resulting from thousands of simultaneous neural processesassociated with different portions of the brain. EEG data can beclassified in various bands. According to various embodiments, brainwavefrequencies include delta, theta, alpha, beta, and gamma frequencyranges. Delta waves are classified as those less than 4 Hz and areprominent during deep sleep. Theta waves have frequencies between 3.5 to7.5 Hz and are associated with memories, attention, emotions, andsensations. Theta waves are typically prominent during states ofinternal focus.

Alpha frequencies reside between 7.5 and 13 Hz and typically peak around10 Hz. Alpha waves are prominent during states of relaxation. Beta waveshave a frequency range between 14 and 30 Hz. Beta waves are prominentduring states of motor control, long range synchronization between brainareas, analytical problem solving, judgment, and decision making Gammawaves occur between 30 and 60 Hz and are involved in binding ofdifferent populations of neurons together into a network for the purposeof carrying out a certain cognitive or motor function, as well as inattention and memory. Because the skull and dermal layers attenuatewaves in this frequency range, brain waves above 75-80 Hz are difficultto detect and are often not used for stimuli response assessment.

However, the techniques and mechanisms of the present inventionrecognize that analyzing high gamma band (kappa-band: Above 60 Hz)measurements, in addition to theta, alpha, beta, and low gamma bandmeasurements, enhances neurological attention, emotional engagement andretention component estimates. In particular embodiments, EEGmeasurements including difficult to detect high gamma or kappa bandmeasurements are obtained, enhanced, and evaluated. Subject and taskspecific signature sub-bands in the theta, alpha, beta, gamma and kappabands are identified to provide enhanced response estimates. Accordingto various embodiments, high gamma waves (kappa-band) above 80 Hz(typically detectable with sub-cranial EEG and/ormagnetoencephalograophy) can be used in inverse model-based enhancementof the frequency responses to the stimuli.

Various embodiments of the present invention recognize that particularsub-bands within each frequency range have particular prominence duringcertain activities. A subset of the frequencies in a particular band isreferred to herein as a sub-band. For example, a sub-band may includethe 40-45 Hz range within the gamma band. In particular embodiments,multiple sub-bands within the different bands are selected whileremaining frequencies are band pass filtered. In particular embodiments,multiple sub-band responses may be enhanced, while the remainingfrequency responses may be attenuated.

An information theory based band-weighting model is used for adaptiveextraction of selective dataset specific, subject specific, taskspecific bands to enhance the effectiveness measure. Adaptive extractionmay be performed using fuzzy scaling. Stimuli can be presented andenhanced measurements determined multiple times to determine thevariation profiles across multiple presentations. Determining variousprofiles provides an enhanced assessment of the primary responses aswell as the longevity (wear-out) of the marketing and entertainmentstimuli. The synchronous response of multiple individuals to stimulipresented in concert is measured to determine an enhanced across subjectsynchrony measure of effectiveness. According to various embodiments,the synchronous response may be determined for multiple subjectsresiding in separate locations or for multiple subjects residing in thesame location.

Although a variety of synthesis mechanisms are described, it should berecognized that any number of mechanisms can be applied—in sequence orin parallel with or without interaction between the mechanisms.

Although intra-modality synthesis mechanisms provide enhancedsignificance data, additional cross-modality synthesis mechanisms canalso be applied. A variety of mechanisms such as EEG, Eye Tracking, GSR,EOG, and facial emotion encoding are connected to a cross-modalitysynthesis mechanism. Other mechanisms as well as variations andenhancements on existing mechanisms may also be included. According tovarious embodiments, data from a specific modality can be enhanced usingdata from one or more other modalities. In particular embodiments, EEGtypically makes frequency measurements in different bands like alpha,beta and gamma to provide estimates of significance. However, thetechniques of the present invention recognize that significance measurescan be enhanced further using information from other modalities.

For example, facial emotion encoding measures can be used to enhance thevalence of the EEG emotional engagement measure. EOG and eye trackingsaccadic measures of object entities can be used to enhance the EEGestimates of significance including but not limited to attention,emotional engagement, and memory retention. According to variousembodiments, a cross-modality synthesis mechanism performs time andphase shifting of data to allow data from different modalities to align.In some examples, it is recognized that an EEG response will often occurhundreds of milliseconds before a facial emotion measurement changes.Correlations can be drawn and time and phase shifts made on anindividual as well as a group basis. In other examples, saccadic eyemovements may be determined as occurring before and after particular EEGresponses. According to various embodiments, time corrected GSR measuresare used to scale and enhance the EEG estimates of significanceincluding attention, emotional engagement and memory retention measures.

Evidence of the occurrence or non-occurrence of specific time domaindifference event-related potential components (like the DERP) inspecific regions correlates with subject responsiveness to specificstimulus. According to various embodiments, ERP measures are enhancedusing EEG time-frequency measures (ERPSP) in response to thepresentation of the marketing and entertainment stimuli. Specificportions are extracted and isolated to identify ERP, DERP and ERPSPanalyses to perform. In particular embodiments, an EEG frequencyestimation of attention, emotion and memory retention (ERPSP) is used asa co-factor in enhancing the ERP, DERP and time-domain responseanalysis.

EOG measures saccades to determine the presence of attention to specificobjects of stimulus. Eye tracking measures the subject's gaze path,location and dwell on specific objects of stimulus. According to variousembodiments, EOG and eye tracking is enhanced by measuring the presenceof lambda waves (a neurophysiological index of saccade effectiveness) inthe ongoing EEG in the occipital and extra striate regions, triggered bythe slope of saccade-onset to estimate the significance of the EOG andeye tracking measures. In particular embodiments, specific EEGsignatures of activity such as slow potential shifts and measures ofcoherence in time-frequency responses at the Frontal Eye Field (FEF)regions that preceded saccade-onset are measured to enhance theeffectiveness of the saccadic activity data.

According to various embodiments, facial emotion encoding uses templatesgenerated by measuring facial muscle positions and movements ofindividuals expressing various emotions prior to the testing session.These individual specific facial emotion encoding templates are matchedwith the individual responses to identify subject emotional response. Inparticular embodiments, these facial emotion encoding measurements areenhanced by evaluating inter-hemispherical asymmetries in EEG responsesin specific frequency bands and measuring frequency band interactions.The techniques of the present invention recognize that not only areparticular frequency bands significant in EEG responses, but particularfrequency bands used for communication between particular areas of thebrain are significant. Consequently, these EEG responses enhance theEMG, graphic and video based facial emotion identification.

Integrated responses are generated at 621. According to variousembodiments, the data communication device transmits data to theresponse integration using protocols such as the File Transfer Protocol(FTP), Hypertext Transfer Protocol (HTTP) along with a variety ofconventional, bus, wired network, wireless network, satellite, andproprietary communication protocols. The data transmitted can includethe data in its entirety, excerpts of data, converted data, and/orelicited response measures. According to various embodiments, data issent using telecommunications, wireless, Internet, satellite, or anyother communication mechanisms that is capable of conveying informationfrom multiple subject locations for data integration and analysis. Themechanism may be integrated in a set top box, computer system, receiver,mobile device, etc.

In particular embodiments, the data communication device sends data tothe response integration system. According to various embodiments, theresponse integration system combines analyzed and enhanced responses tothe stimulus material while using information about stimulus materialattributes. In particular embodiments, the response integration systemalso collects and integrates user behavioral and survey responses withthe analyzed and enhanced response data to more effectively measure andtrack neuro-responses to stimulus materials. According to variousembodiments, the response integration system obtains attributes such asrequirements and purposes of the stimulus material presented.

Some of these requirements and purposes may be obtained from a varietyof databases. According to various embodiments, the response integrationsystem also includes mechanisms for the collection and storage ofdemographic, statistical and/or survey based responses to differententertainment, marketing, advertising and otheraudio/visual/tactile/olfactory material. If this information is storedexternally, the response integration system can include a mechanism forthe push and/or pull integration of the data, such as querying,extraction, recording, modification, and/or updating.

The response integration system can further include an adaptive learningcomponent that refines user or group profiles and tracks variations inthe neuro-response data collection system to particular stimuli orseries of stimuli over time. This information can be made available forother purposes, such as use of the information for presentationattribute decision making. According to various embodiments, theresponse integration system builds and uses responses of users havingsimilar profiles and demographics to provide integrated responses at621. In particular embodiments, stimulus and response data is stored ina repository at 623 for later retrieval and analysis.

According to various embodiments, various mechanisms such as the datacollection mechanisms, the intra-modality synthesis mechanisms,cross-modality synthesis mechanisms, etc. are implemented on multipledevices. However, it is also possible that the various mechanisms beimplemented in hardware, firmware, and/or software in a single system.FIG. 7 provides one example of a system that can be used to implementone or more mechanisms. For example, the system shown in FIG. 7 may beused to implement a data analyzer.

According to particular example embodiments, a system 700 suitable forimplementing particular embodiments of the present invention includes aprocessor 701, a memory 703, an interface 711, and a bus 715 (e.g., aPCI bus). When acting under the control of appropriate software orfirmware, the processor 701 is responsible for such tasks such aspattern generation. Various specially configured devices can also beused in place of a processor 701 or in addition to processor 701. Thecomplete implementation can also be done in custom hardware. Theinterface 711 is typically configured to send and receive data packetsor data segments over a network. Particular examples of interfaces thedevice supports include host bus adapter (HBA) interfaces, Ethernetinterfaces, frame relay interfaces, cable interfaces, DSL interfaces,token ring interfaces, and the like.

In addition, various very high-speed interfaces may be provided such asfast Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces,HSSI interfaces, POS interfaces, FDDI interfaces and the like.Generally, these interfaces may include ports appropriate forcommunication with the appropriate media. In some cases, they may alsoinclude an independent processor and, in some instances, volatile RAM.The independent processors may control such communications intensivetasks as data synthesis.

According to particular example embodiments, the system 700 uses memory703 to store data, algorithms and program instructions. The programinstructions may control the operation of an operating system and/or oneor more applications, for example. The memory or memories may also beconfigured to store received data and process received data.

Because such information and program instructions may be employed toimplement the systems/methods described herein, the present inventionrelates to tangible, machine readable media that include programinstructions, state information, etc. for performing various operationsdescribed herein. Examples of machine-readable media include, but arenot limited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks and DVDs;magneto-optical media such as optical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory devices (ROM) and random access memory (RAM).Examples of program instructions include both machine code, such asproduced by a compiler, and files containing higher level code that maybe executed by the computer using an interpreter.

Although the foregoing invention has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications may be practiced within the scope of theappended claims. Therefore, the present embodiments are to be consideredas illustrative and not restrictive and the invention is not to belimited to the details given herein, but may be modified within thescope and equivalents of the appended claims.

1. A method, comprising: obtaining stimulus material for presentation toa subject; generating a virtual reality environment for the stimulusmaterial; integrating the virtual reality environment and the stimulusmaterial; exposing the subject to the stimulus material in the virtualreality environment; collecting neuro-response data from the subjectexposed to the stimulus material in the virtual reality environment,wherein the neuro-response data includes electroencephalography data;wherein the neuro-response data from the subject is analyzed todetermine an effectiveness of the stimulus material in the virtualreality environment, wherein the stimulus material comprises marketingmaterial.
 2. The method of claim 1, wherein the neuro-response data isused to dynamically modify the stimulus material in the virtual realityenvironment.
 3. The method of claim 1, wherein the neuro-response datais used to dynamically modify the virtual reality environment.
 4. Themethod of claim 1, wherein interaction data is received from the subjectto manipulate the stimulus material.
 5. The method of claim 1, whereinthe stimulus material is a product or product package.
 6. The method ofclaim 1, wherein the stimulus material is a service or offering.
 7. Themethod of claim 1, wherein the stimulus material is a sign or placard.8. The method of claim 1, wherein the neuro-response data is collectedusing a plurality of modalities including electroencephalography (EEG)and electrooculography (EOG).
 9. The method of claim 1, whereinanalyzing the neuro-response data comprises obtaining target anddistracter event related potential (ERP) measurements to determinedifferential measurements of ERP time domain components at multipleregions of the brain (DERP).
 10. The method of claim 9, whereinanalyzing the neuro-response data further comprises obtaining eventrelated time-frequency analysis of the differential measurements toassess attention, emotion and memory retention (DERPSPs) across multiplefrequency bands.
 11. A system, comprising: an interface configured toobtain stimulus material for presentation to a subject; a processorconfigured to generate a virtual reality environment for the stimulusmaterial and integrate the virtual reality environment and the stimulusmaterial; a display configured to expose the subject to the stimulusmaterial in the virtual reality environment; electroencephalography(EEG) electrodes configured to collect neuro-response data from thesubject exposed to the stimulus material in the virtual realityenvironment; and a data analyzer configured to evaluate theneuro-response data from the subject to determine an effectiveness ofthe stimulus material in the virtual reality environment, wherein thestimulus material is marketing material.
 12. The system of claim 11,wherein the neuro-response data is used to dynamically modify thestimulus material in the virtual reality environment.
 13. The system ofclaim 11, wherein the neuro-response data is used to dynamically modifythe virtual reality environment.
 14. The system of claim 11, whereininteraction data is received from the subject to manipulate the stimulusmaterial.
 15. The system of claim 11, wherein the stimulus material is aproduct or product package.
 16. The system of claim 11, wherein thestimulus material is a service or offering.
 17. The system of claim 11,wherein the stimulus material is a sign or placard.
 18. The system ofclaim 11, wherein evaluating the neuro-response data comprises obtainingtarget and distracter event related potential (ERP) measurements todetermine differential measurements of ERP time domain components atmultiple regions of the brain (DERP).
 19. The system of claim 18,wherein evaluating the neuro-response data further comprises obtainingevent related time-frequency analysis of the differential measurementsto assess the attention, emotion and memory retention (DERPSPs) acrossmultiple frequency bands.
 20. An apparatus, comprising: means forobtaining stimulus material for presentation to a subject; means forgenerating a virtual reality environment for the stimulus material;means for integrating the virtual reality environment and the stimulusmaterial; means for exposing the subject to the stimulus material in thevirtual reality environment; and means for collecting neuro-responsedata from the subject exposed to the stimulus material in the virtualreality environment, wherein the neuro-response data compriseselectroencephalography data; wherein the neuro-response data from thesubject is analyzed to determine the effectiveness of stimulus materialin the virtual reality environment, wherein the stimulus material ismarketing material.